SetPINNs: Set-based Physics-informed Neural Networks
Please login to view abstract download link
Physics-Informed Neural Networks (PINNs) have gained prominence as a mesh-free, data-driven approach to solving partial differential equations (PDEs). However, conventional PINNs, which typically employ multilayer perceptrons (MLPs) with point-wise predictions, often fail to capture critical spatial and temporal dependencies within physical systems. This limitation can lead to suboptimal performance, such as overly smooth or trivial solutions, especially in systems with high-frequency or multi-scale features. To address this, we propose SetPINNs, a novel framework inspired by Finite Element Methods (FEM). SetPINNs discretize the domain and group points from elements into sets, allowing the model to learn dependencies among neighboring points and incorporate physical constraints using a set-wise physics loss. This approach, which leverages attention mechanisms to model dependencies invariant to input permutations, outperforms existing PINN methods across various physical systems. In extensive experiments on synthetic and real-world problems, including predicting molecular activity coefficients and agglomerate breakage, SetPINNs show superior generalization and faster convergence, while avoiding trivial solutions. Our findings suggest that SetPINNs provide a robust alternative for solving complex PDEs, extending the applicability of machine learning to more challenging physical domains. These results offer significant contributions to both machine learning-based PDE solvers and the broader field of numerical methods, advancing the integration of physics constraints in deep learning frameworks.
